A charged particle beam device is a device that irradiates a specimen with a charged particle beam such as an electron beam and an ion beam emitted from a charged particle source. Such a device includes a device including an irradiation optical system column (a lens-barrel) and an imaging optical system column. Patent Literature 1 discloses an electron microscope including two columns of an irradiation optical system and an imaging optical system. Further, Patent Literature 1 describes that a specimen is irradiated with ultraviolet rays to charge the specimen, and the specimen in a charged state is irradiated with an electron beam to detect defects. Patent Literature 2, similarly to Patent Literature 1, discloses an electron microscope including two optical systems. Further, Patent Literature 2 describes an evaluation method to evaluate an optical performance of an imaging optical system by arranging a Wien filter and an inspection chart and using electrons emitted from the inspection chart. The Wien filter separates an electron beam with which a specimen is irradiated off a trajectory of electrons emitted from the specimen. The inspection chart autonomously emits an electron beam for inspection. The inspection chart is arranged on a position of the specimen that is originally an irradiation object of the electron beam to perform an optical axis adjustment of the imaging optical system.